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A  relatively  new  ocean  surveillance  system,  the  Inter-array  Processor  II,  has  been 
developed  to  an  operational  stage.  Originally,  training  was  to  consist  of  on-the-job 
exposure  to  actual  equipment  with  training  manuals  to  guide  the  student.  However, 
because  of  high  equipment  costs  and  limited  availability,  the  feasibility  of  using  a 
computer-based,  optical-disc  trainer  as  an  alternative  instructional  delivery  system  is 
being  investigated  by  the  Naval  Ocean  Systems  Center  (NOSC).  This  report  provides 
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recommendations  to  NOSC  for  the  selection  of  the  training  scenarios,  instructional 
delivery  strategies,  student/training  system  interface,  and  testing  and  feedback 
requirements,  along  with  a  discussion  of  the  rationale  behind  the  recommendations. 


FOREWORD 


This  research  and  development  was  conducted  under  subproject  PE  62759  (Human 
Factors  Technology  Development)  and  was  sponsored  by  the  Naval  Electronic  Systems 
Command  (NAVELEX).  The  objective  of  this  R<ScD  was  to  provide  training 
recommendations  to  the  Naval  Ocean  Systems  Center  (NOSC)  in  support  of  its  effort  to 
test  the  feasibility  of  using  optical-disc  hardware  in  developing  a  portable  operator 
training  device. 

The  results  of  this  work  are  primarily  intended  for  NOSC  and  NAVELEX.  They  should 
also  be  of  interest  to  instructional  and  equipment  development  agencies  that  are  tasked 
with  developing  on-site  operator  trainers. 


1.  W.  RENARD  3.  R.  T^EEDDALE 

Captain,  U.S.  Navy  Technical  Director 

Commanding  Officer 
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SUMMARY 


Problem 


The  integrated  undersea  surveillance  system  (IUSS),  which  is  being  developed  to 
increase  the  accuracy  and  speed  of  acoustic  target  processing,  is  comprised  of  a  number 
of  detection,  tracking,  and  reporting  devices.  One  of  these  subsystems  is  known  as  the 
Inter-array  Processor  I!  (IAP-II).  The  design  of  1AP-II  has  recently  stabilized  sufficiently 
to  allow  the  development  of  training  materials. 

The  original  training  {dan  calls  for  on-the-job  training  using  operational  system 
equipment.  However,  because  of  equipment  cost  and  availability  restrictions,  the 
feasibility  of  using  a  computer-based,  optical-disc  trainer  as  an  alternate  delivery  system 
for  operator  training  is  being  investigated  by  the  Naval  Ocean  Systems  Center  (N05C). 

Objective 

The  purpose  of  this  effort  was  to  provide  training  recommendations  to  NOSC  in 
support  of  their  development  of  the  optical-disc  trainer. 

Approach  and  Results 

The  following  issues  were  addressed  and  recommendations  made  for  each: 

1 .  T raining  scenarios. 

2.  Instructional  delivery  strategies. 

3.  Student/system  interface  requirements. 

4.  Testing  and  feedback  requirements. 
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INTRODUCTION 


Problem 


Navy  technical  personnel  tasked  with  performing  ocean  surveillance  are  required  to 
be  proficient  at  operating  a  variety  of  equipment  included  in  the  integrated  undersea 
surveillance  system  (1USS).  1USS  is  comprised  of  a  number  of  detection,  tracking,  and 
reporting  devices  that  are  designed  to  increase  the  accuracy  and  speed  of  acoustic  target 
processing.  1USS  components  are  being  developed,  modified,  and  brought  on  station  over  a 
period  of  time.  Initial  training  on  these  components  is  most  often  provided  to  personnel 
on-the-job,  using  actual  system  equipment.  In  many  cases,  this  equipment  is  expensive, 
needed  for  performing  operational  tasks,  and  not  designed  to  be  used  as  an  operator 
training  device.  The  use  of  relatively  low-cost  hardware  simulators  is  now  being 
investigated  as  an  alternative  instructional  delivery  system  for  operator  training. 

One  IUSS  subsystem,  the  Inter-array  Processor  I!  (IAP-II),  has  gone  through  a  number 
of  system  design  changes.  In  the  last  few  years,  the  design  stabilized  sufficiently  to  allow 
for  the  development  of  training  materials.  The  original  training  plan  called  for  on-the-job 
training  using  actual  system  equipment.  However,  because  of  the  high  equipment  and 
operational  costs  associated  with  using  an  operational  IAP-II  system  as  an  on-site  training 
device,  the  feasibility  of  using  a  computer-based,  optical-disc  trainer  as  an  alternate 
instructional  delivery  system  is  being  investigated  by  the  Naval  Ocean  Systems  Center 
(NOSC).  The  optical  disc  will  provide  the  images  presently  generated  in  real  time  by  the 
IAP-I!  system  and  the  computer  will  provide  text  allowing  the  student/operator  to  query 
the  system,  as  well  as  give  feedback.  The  training  requirements  for  an  optical-disc 
trainer  need  to  be  determined  prior  to  the  hardware  development  phase. 

Background 

The  Naval  Electronics  System  Command  (NAVELEX  1 24)  tasked  a  contractor  to 
develop  training  materials  for  use  with  the  on-site  IAP-II  system.  The  instructional 
materials  that  were  developed  under  that  tasking  consist  of  a  series  of  training  packets 
for  use  on  the  job  and  require  the  dedication  of  an  I^P  display  console  to  conduct  the 
training.  They  present  all  of  the  concepts  and  procedures  necessary  to  operate  the  IAP-II. 
All  of  the  hands-on  practice  requires  the  use  of  the  actual  equipment.  The  training  is 
self-paced,  expository  instruction  with  limited  requirement  for  instructor 
supervision/interaction.  Each  packet  covers  a  different  aspect  of  IAP-II  operation,  with 
some  review  as  the  series  progresses  to  aid  in  the  maintenance  of  operator  skills.  The 
packets  are  preceded  by  a  reading  assignment  and  require  the  operator  to  answer  a  series 
of  open-ended  questions  about  IAP-II  operating  characteristics.  The  operator  then  follows 
instructions  to  sign  on  to  the  operational  equipment  and  perform  the  operations  called  for 
in  each  packet.  Each  packet  supports  operational  procedures  that  are  performed  in  real 
time  using  actual  equipment. 

There  are  both  advantages  and  disadvantages  to  using  the  IAP-II  system  for  training. 
Advantages  include  high  task  fidelity  and  minimized  negative  skill  transfer.  However, 
there  are  five  distinct  disadvantages  that  must  be  considered  with  the  use  of  operational 
equipment: 

1.  Some  critical  operations  cannot  be  performed  because  the  console  is  "live”  and 
thus  capable  of  confusing  the  operational  environment  by  sending  training  data  that  may 
be  perceived  as  actual  target  data. 
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2.  The  range  of  practice  situations  the  student  can  be  exposed  to  is  limited. 

3.  Operating  in  real  time  precludes  demonstration  of  procedures  that  take  too  long 
and  require  too  much  computing  power. 

4.  The  costs  associated  with  dedicating  major  on-site  systems  for  training  purposes 
can  be  prohibitive. 

5.  Perhaps  most  important  in  a  training  situation,  it  is  difficult  to  monitor  student 
performance  and  provide  high  quality,  diagnostic  feedback  to  the  student  when  operation¬ 
al  equipment  is  used  as  a  training  device.  This  difficulty  is  due  primarily  to  an  inability  to 
modify  the  software  in  the  operational  device  to  support  training  requirements. 

Use  of  a  training  simulator  as  an  instructional  delivery  system  can  provide  many 
benefits.  Because  the  simulator  does  not  involve  any  operational  equipment,  training  does 
not  interfere  with  equipment  used  for  normal  watches.  The  simulator  allows  practice  on 
problems  not  normally  found  in  routine  situations,  allows  event  time  to  be  compressed  for 
practice,  and  provides  monitoring  of  student/operator  performance.  In  addition,  the  costs 
are  much  lower.  The  benefits,  especially  the  ability  to  monitor  performance  and  give 
feedback,  argue  strongly  for  the  use  of  a  simulator. 

The  capability  to  simulate  the  system  environment  that  is  offered  by  the  computer- 
based,  optical-disc  trainer  could  maximize  the  exposure  of  trainees  to  a  wide  range  of 
problems  for  practice,  reduce  skill  transfer  problems,  and  limit  use  of  dedicated  on-site 
systems,  as  well  as  decrease  the  costs  associated  with  building  and  maintaining  training 
systems.  However,  to  maximize  the  effectiveness  of  the  optical  disc  device  as  an 
operator  trainer,  the  operator  training  requirements  need  to  be  established  and  incorpo¬ 
rated  into  the  final  system. 

Objective 

The  purpose  of  this  effort  was  to  provide  training  recommendations  to  NIOSC  in 
support  of  its  development  of  a  computer-based,  optical-disc  instructional  delivery  system 
for  IAP-II  operator  training. 


APPROACH 

Four  training  issues  were  addressed  and  recommendations  given  for  each:  (1)  training 
scenarios,  (2)  instructional  delivery  strategies,  (3)  student/system  interface  requirements, 
and  (4)  testing  and  feedback  requirements. 

RESULTS 


Training  Scenarios 

Selection  Criteria 


The  IAP-II  training  package  being  developed  will  consist  of  approximately  30  training 
packets.  However,  at  the  time  this  project  was  initiated,  only  four  were  available  for 
review.  Because  the  purpose  of  this  effort  was  to  demonstrate  the  feasibility  of  using  a 
relatively  low-cost,  computer-based  optical-disc  simulator  as  an  instructional  delivery 
system,  it  was  decided  to  select  the  packet  that  incorporated  the  widest  range  of  operator 
skills. 
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Recommendation 


NOSC  should  use  the  1AP-II  Training  Packet  2  to  demonstrate  the  performance  of  a 
low-cost,  comouter-based,  optical-disc  trainer  as  an  instructional  delivery  device.  Of  the 
four  packets,  it  has  the  largest  number  of  representative  procedures  and  requires  the 
student  to  use  all  input  devices.  Further,  the  packet  text  and  instructions  need  only  minor 
modifications  for  use  in  the  alternate  instructional  system. 

Instructional  Delivery  Strategies 

Alternate  Strategies 

IAP-II  contains  two  CRT  displays  that  can  be  used  for  alphanumerics  and  several 
types  of  graphic  displays,  as  well  as  a  keyboard,  touch  panel,  and  a  trackball.  1AP-1I 
Training  Packet  2  requires  the  use  of  all  of  these  devices.  Training  requirements  place 
additional  demands  on  these  resources  and/or  require  additional  hardware  and  software. 
There  are  a  number  of  configurations  possible  in  designing  the  simulator  hardware.  Five 
alternate  strategies  are  summarized  in  Table  1.  They  include  (1)  an  additional  CRT 
devoted  exclusively  to  text,  (2)  text  windows  on  the  operational  CRTs,  (3)  hard-copy  text, 
(4)  auditory  texts,  and  (5)  adjunct  information.  For  each  strategy,  it  is  assumed  that  a 
keyboard  is  available,  as  well  as  either  a  mouse  or  a  touch  panel  and  trackball. 

Selection  of  the  optimum  strategy  was  based  on  three  considerations: 

1.  The  number  and  variety  of  things  to  which  the  student  must  pay  attention, 
including  the  number  of  displays,  the  amount  of  information,  the  number  of  times  the 
student  must  look  back  and  forth  between  the  displays,  and  the  distance  between  displays, 
must  be  minimized. 

2.  Conceptually  complex  information  must  be  presented  in  written  form,  particu¬ 
larly  if  the  information  must  be  referred  to  later  (i.e.,  for  review  and  study). 

3.  Oral  information  should  be  limited  to  short,  simple  instructions  unless  it  is  a 
repetition  of  material  presented  earlier  in  text  format. 

Of  the  alternative  strategies  listed  in  Table  1,  number  2  offers  the  greatest  number 
of  training  presentation  modes  and  allows  the  student  to  examine  the  text  in  whatever 
mode  desired.  It  calls  for  two  CRTs  with  windowing  capability  for  written  text,  a  text 
handbook,  and  an  audio  text.  Use  of  a  text  window  allows  the  student  to  focus  on  the 
normal  IAP  display  terminals  for  operator  functions  and  training  text  without  needing  a 
third  CRT.  This  precludes  the  need  to  look  back  and  forth  between  the  text  on  a  third 
CRT  and  the  normal  displays. 
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Table  1 


Alternate  Instructional  Delivery 
Support  Strategies 


Strategy 

CRTs— Number 
and  Use 

Presentation 

Modes 

Student 

Input  Devices 

1 

Two  for  normal  displays 
One  for  training  text 

Text  handbook 

Third  CRT  audio  cue 

Keyboard;  track¬ 
ball  and  touch  panel 
or  mouse 

2 

Two  for  normal  displays 
with  windowing  for  text 

Text  bar  ook 

Audio  c 

Keyboard;  track¬ 
ball  and  touch  panel 
or  mouse 

3 

Two  for  normal  displays 
with  windowing  for  text 

Text  )  Ibook 

Audio  ,  mt  cues 

Keyboard;  track¬ 
ball  and  touch  panel 
or  mouse 

4 

Two  for  normal  displays 
with  windowing  for  cues 

Written  text  only 

Keyboard;  track¬ 
ball  and  touch  panel 
or  mouse 

5 

Two  for  normal  displays 

Audio  text 

Text  handbook 

Keyboard;  track¬ 
ball  and  touch  panel 
or  mouse 

Strategy  2  includes  the  recommendation  that  the  text  should  be  printed  as  a 
handbook.  This  handbook  would  provide  a  number  of  benefits  to  the  student.  First,  hard¬ 
copy  text  in  handbook  form  would  enable  the  student  to  read  and  review  material  specific 
to  the  frame  of  interest,  as  well  as  review  text  from  previous  frames.  In  past  studies  at 
NAVPERSRANDCEN,1  2  it  was  found  that  students  take  considerably  longer  to  complete 
training  when  they  cannot  review  the  materials  at  their  leisure.  Second,  allowing  the 
students  to  retain  the  handbook  following  the  course  allows  them  to  review  the  material 
when  needed  on  the  job,  thus  ensuring  the  availability  of  refresher  training  materials. 

Finally,  under  strategy  2,  it  is  recommended  that  audio  text  be  used  in  two  ways. 
First,  it  can  be  used  to  give  short  instructions  (i.e.,  press  specified  button,  look  at  specific 
area  on  CRT).  Second,  if  the  oral  text  is  the  same  as  the  written  text,  the  student  can 
listen  to  the  instructions/rationale  while  looking  at  the  appropriate  display.  This  would 
preclude  the  student  from  having  to  shift  attention  from  reading  to  looking  and  would 
allow  greater  concentration  on  the  display  of  interest.  Because  using  audio  only  is  not 
considered  suitable  for  complex  or  long  messages,  it  should  not  be  used  in  isolation.  With 
strategy  2,  the  student  can  use  either  or  both  of  the  presentation  modes. 


‘Hurlock,  R.  E.,  Slough,  D.  A.  Experimental  evaluation  of  PLATO  IV  technology:  Final 
report  (NPRDC  Tech.  Rep.  76TQ-44).  San  Diego:  Personnel  Research  and  Development 
Center,  August  1976.  (AD-A029  384) 

2Stern,  H.  W.  Transfer  of  training  following  computer-based  instruction  in  basic 
oscilloscope  procedures  (NPRDC  Tech.  Rep.  76-1).  San  Diego:  Navy  Personnel 
Research  and  Development  Center,  3uly  197  5.  (AD-A012  637) 
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Strategy  1  is  considered  the  least  desirable.  Students  would  have  to  constantly 
switch  their  attention  from  the  operational  CRTs  to  the  text  screen.  It  might  be  possible 
to  provide  audio  cues  to  overcome  some  of  this  problem,  but  it  provides  the  least 
effective  solution  to  the  training  problem.  The  other  strategies  are  considered  to  be 
approximately  equal  in  value  except,  possibly,  for  strategy  5,  if  it  could  be  configured 
with  cheaper  CRTs  not  requiring  the  window  feature.  This  would  reduce  the  cost  and 
might  make  this  option  attractive.  There  is  no  evidence  that  reading  text  from  the  CRT 
imoroves  learning  over  reading  text  from  hard  copy.  In  fact,  as  noted  above,  CRT-only 
text  may  increase  learning  time. 

Recommendation 


The  instructional  delivery  strategy  selected  should  have  the  following: 

1.  Two  CRTs  with  windowing  capability  for  text  display. 

2.  Written  text,  in  hard-copy  form. 

3.  Audio  supplements  to  the  text  (desirable  but  not  essential).  If  an  audio 
supplement  is  included,  the  following  criteria  should  be  met: 

a.  Audio  should  be  used  only  when  the  message  is  simple  or  when  the  message 
requires  immediate  action. 

b.  Audio  should  be  an  option  to  the  student. 

c.  Audio  text  should  duplicate  the  written  text. 

d.  Audio  text  repetitions  should  be  available  to  the  student. 

4.  Student  input  devices,  including  keyboard,  touch  panel,  and  trackball. 
Student/System  Interface  Requirements 

Instructional  Issues 

As  indicated  previously,  the  IAP-II  console  contains  two  CRT  displays  that  can  be 
used  for  alphanumerics  and  several  types  of  graphic  displays,  a  keyboard,  a  touch  panel, 
and  a  trackball.  The  computer-based,  optical-disc  training  simulator  is  being  designed  to 
emulate  the  actual  console  as  closely  as  possible.  Since  the  original  and  the  simulator 
systems  are  both  designed  to  minimize  the  number  of  operator  responses  to  achieve  a 
given  display,  the  main  inputs  must  be  made  through  the  touch  panel  and  trackball.  The 
keyboard  is  activated  only  when  needed,  thereby  reducing  the  amount  of  typing  required 
of  the  operator.  Since  the  student  will  use  all  available  spaces  in  the  touch  panel  at  some 
time  during  Packet  2  training,  some  other  means  of  allowing  the  student/operator  access 
to  training  text  and  other  aids  is  needed  so  that  he  or  she  can  easily  move  through  the 
training  material. 

Originally,  Packet  2  was  a  paper  workbook  intended  to  be  used  with  live  equipment; 
however,  the  optical-disc  trainer  will  have  the  Packet  2  text  available  on  one  CRT  and  an 
option  for  a  paper  copy.  Student  actions  are  directed  at  all  times  in  Packet  2  for  both 
versions.  Most  entries  are  made  through  the  touch  panel,  which  is  the  normal  mode  of 
system  operation.  The  keyboard  is  used  to  enter  the  operator's  name,  array  number, 


time/date  group,  beam  number,  and  beam  frequency.  The  trackball  is  used  to  position  the 
cursor.  For  training  purposes,  some  means  is  needed  to  allow  the  student  to  manipulate 
training  text  independent  of  the  normal  displays.  The  student  will  normally  progress  from 
frame  to  frame  in  a  linear  sequence,  but  some  student  flexibility  is  desirable.  A  provision 
is  necessary  to  allow  for  the  student  to  move  forward  or  backward  a  few  frames  or  to 
repeat  a  section  of  the  training. 

Student  Interfaces.  The  way  students  interface  with  the  training  system  is  critical. 
They  must  have  a  minimum  of  choices  that  are  clearly  marked.  One  possibility  for  the 
future  would  be  the  addition  of  another  touch  panel.  However,  at  this  time,  the  best 
interface  can  be  constructed  with  the  keyboard. 

The  student  must  be  able  to  control  the  following  five  variables:  (1)  frame  duration, 
(21  training  entry  point,  (3)  training  exit  point,  (4)  review  (single  frame  and  section),  and 
(5)  access  to  "help."  Control  of  these  variables  requires  software  to  allow  student  access 
to  a  selected  set  of  keys  that  remain  constant  for  each  variable.  For  example,  if  arrow 
keys  are  available,  they  can  be  used  to  move  forward  and  backward  in  the  text  when  the 
student  wants  to  review.  They  should  be  keys  that  are  not  used  normally  so  that  they  can 
be  relabeled  with  the  appropriate  term.  The  text  must  be  linked  to  the  nominal  display 
functions,  but  some  provision  is  needed  to  allow  the  student  to  move  forward  or  backward 
a  few  frames  of  IAP-I1  displays  without  changing  the  text.  This  is  necessary  to  provide 
feedback  to  the  student  on  the  consequences  of  the  action  performed.  For  example,  when 
some  touch  panel  action  is  performed,  the  IAP-II  frame  should  be  advanced  while  the  text 
is  held  constant.  When  the  student  is  satisfied  with  the  result,  a  keyboard  function  will 
allow  advancement  to  the  next  text  frame  describing  the  required  action  and  any 
comments  on  the  action.  To  aid  in  ensuring  the  correct  touch  panel  response,  the  correct 
square  should  be  blinking.  The  student  should  be  able  to  toggle  off  both  the  blinking  touch 
panel  and  the  text  to  allow  self-testing  of  procedural  knowledge. 

Advance  organizers.  The  training  packets  provide  little  advance  information  as  to 
what  is  expected  of  the  student.  The  current  packet  includes  material  within  the  packet 
that  is  used  as  a  review  of  the  completed  lesson.  This  material  could  easily  be  moved  to 
the  beginning  of  the  training  and  thus  provide  the  student  with  an  overview  of  what  is 
expected  in  that  lesson. 

Menu.  At  least  one  line  at  the  bottom  of  the  CRT  should  be  reserved  to  present  a 
menu  for  the  special  keys.  Another  menu  is  needed  to  mark  major  points  in  the  training 
exercises,  such  as  the  advance  organizers.  This  allows  the  student  to  jump  out  of  the 
normal  sequence  to  review  the  overall  objective  or  other  information  that  might  be 
needed.  If  the  student  does  not  complete  the  lesson  in  one  session,  some  means  of 
returning  to  the  break  point  is  needed.  One  way  to  do  this  is  to  provide  a  menu  of  the 
major  and  semi-major  procedures  in  Packet  2,  which  would  allow  the  student  to  return  to 
the  approximate  stop  point.  Menu  selection  should  be  either  by  moving  the  curser,  as  in 
normal  IAP  operation,  or  by  keyboard  press  of  the  first  letter  of  the  selected  menu  word. 

Text  delivery  and  student  control.  Text  will  be  presented  in  the  bottom  half  of  the 
right-hand  CRT  at  all  times.  In  most  cases,  this  will  not  interfere  with  normal  screen 
functions.  However,  on  a  few  frames,  the  text  will  have  to  alternate  with  the  screen 
graphics.  This  should  be  under  student  control.  Advance  organizers  could  use  the  whole 
CRT,  if  necessary. 

The  simulator  should  be  supplemented  with  a  written  text  that  gives  the  procedures 
for  equipment  operation.  When  there  is  no  provision  for  the  student  to  retain  a  copy  of 
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the  procedures,  they  tend  to  take  copious  notes  for  review  at  a  later  time.  The  hard  copy 
also  ensures  that  the  student  gets  the  correct  procedures  for  later  use  in  the  fleet. 

Text  changes  and  student  actions  will  be  under  separate  control,  but  they  need  to  be 
linked  by  some  software  connection,  a  frame  numbering  sequence,  or  both.  This  means 
the  student  will  have  to  advance  the  IAP-II  frame  separately  from  the  training  text 
frame.  This  should  be  limited  to  a  few  frames.  Some  means  of  linking  the  nominal  IAP-H 
frame  and  the  corresponding  text  frame  has  to  be  provided.  This  is  necessary  if  the 
student  becomes  confused  or  if  the  student  desires  to  jump  out  to  another  part  of  the 
text.  This  is  probably  the  most  difficult  implementation  issue  and  may  require  revision 
during  development  and  during  the  pilot  test  phase. 

Error  and  help  messages.  Error  messages  are  under  software  control.  For  this  initial 
effort,  two  kinds  are  needed:  (1)  a  simple  message  that  the  wrong  function  was  selected, 
and  (2)  an  answer-judging  routine  to  display  a  message  when  an  incorrect  answer  is  typed 
in  or  the  trackball  is  slewed  to  the  wrong  position.  In  both  cases,  the  correct  button 
press,  data  entry,  or  cursor  position  can  be  reiterated  in  the  lower-right  CRT,  perhaps 
with  reverse  video  and  audible  beep  to  ensure  student  attention. 

Help  messages  are  under  student  control.  The  menu  needs  a  provision  for  help. 
These  messages  will  be  displayed  in  the  same  area  reserved  for  text.  As  with  error 
messages,  two  types  are  required:  One  to  provide  students  with  explanations  for  student 
interface  mechanisms  (e.g.,  control  keys,  directions  for  returning  to  previously  seen  text 
for  review,  and  terminology  definitions)  and  the  other,  to  provide  expanded  explanations 
of  operations  and  terms  or  offer  brief  tutorials  on  related  topics. 

Recommendation 


The  student/system  interface  selected  should  have  the  following  characteristics: 

1.  Student  actions  should  be  input  via  the  keyboard. 

2.  Video  text  delivery  should  be  placed  on  the  lower  half  of  the  right-hand  CRT. 

3.  Student  progress  should  be  linear,  with  no  provision  for  "free  play." 

4.  The  bottom  row  of  the  right-hand  CRT  should  be  reserved  for  a  menu. 

5.  Correct  touch  panel  choices  should  blink. 

6.  Student  should  be  able  to  toggle  off  text  and  other  student  prompts. 

7.  Two  types  of  error  messages— "wrong  function"  and  "incorrect  answer"— should 
be  provided  to  the  student. 

8.  Students  should  be  provided  with  interface  explanations  via  student-controlled 
"help"  messages. 

9.  To  ensure  coordination,  some  provision  needs  to  be  developed  for  linking  display 
frames. 


7 


Testing  and  Feedback  Requirements 
Issues 


Assumed  level  of  knowledge.  Packet  2  assumes  some  prior  experience  with  the  IAP 
console  and  requires  preliminary  study  of  reference  materials.  Some  general  questions 
are  asked  before  the  student  signs  on  the  system.  Provision  needs  to  be  made  for 
pretesting  and  for  periodic  reviews  of  student  progress. 

Packet  2  presently  provides  open-ended  pretest  questions  and  one  short  review 
question.  These  questions  could  be  modified  with  minor  effort  to  provide  an  interactive 
review  for  the  students,  but  additional  material  needs  to  be  added  to  inform  them  of  their 
progress.  There  is  also  considerable  pretest  and  review  material  in  Packet  1  that  should 
be  understood  by  the  student  prior  to  undertaking  the  training  in  Packet  2.  If  the  student 
does  not  have  the  basic  skills  and  knowledges  that  are  aquired  through  Packet  1,  poor 
performance  in  Packet  2  might  reflect  this  lack  of  skills  and  knowledges  rather  than  the 
simulation's  training  ability. 

Performance  measurement.  Student  performance  levels  should  be  assessed  at  the 
start  of  training  to  build  a  data  base  that  can  be  used  to  identify  entry-level  students  and 
to  adjust  training  progression.  Progress  tests  should  be  an  accurate  reflection  of  the 
performance  required.  For  example,  in  Packet  1,  the  student  is  asked  to  list  the 
parameters  of  the  "initial  target  state."  These  parameters  do  not  need  to  be  memorized 
because  the  IAP-II  console  prompts  the  operator  for  the  values.  Without  performance 
measures,  evaluation  of  a  computer-based,  optical-disc  training  sytem  is  limited. 

Prompting.  Training  should  be  designed  to  provide  heavy  prompting  at  the  beginning, 
with  decreased  prompting  as  the  student  progresses.  Training  Packet  2  is  designed  to 
prompt  the  student.  As  the  student  becomes  more  familiar  with  the  sequences,  the 
prompts  should  fade.  For  this  pilot  study,  a  temporary  solution  for  the  problem  would  be 
to  allow  the  student  to  toggle  the  prompts  as  necessary.  The  feedback  or  error  messages 
should  be  available  at  all  times  and  not  under  student  control.  The  student  should  perform 
the  task  as  it  is  performed  on  the  job  and  performance  should  be  assessed  based  on  job 
criteria. 

Student  trails.  As  mentioned  previously,  the  student  will  progress  through  the 
training  Packet  frame  by  frame.  This  means  the  information  will  be  presented  in  a  linear 
sequence.  However,  since  the  student  will  be  allowed  to  repeat  difficult  frames  and 
whole  sections,  a  student  audit  trail  is  needed  to  provide  a  record  of  student  progress 
through  text  passages.  This  will  allow  determination  of  difficult  operations.  Since  later 
versions  of  the  training  will  allow  for  branching,  a  provision  for  auditing  these  trails  can 
be  developed  now,  if  time  permits.  For  this  study,  the  following  summary  data  are 
desirable:  (1)  total  time,  (2)  time  for  each  task  or  objective,  (3)  correct  response  rates  for 
each  item  on  the  first  attempt,  and  (4)  number  of  incorrect  responses  to  each  item. 

Recommendation 


The  list  of  testing  and  feedback  requirements  should  include  the  following: 

1.  Pretesting  is  required  to  establish  minimum  entry  level. 

2.  Testing  of  all  objectives  is  needed  to  assess  student  proficiency. 

3.  Prompts  should  be  made  available  to  the  student  when  requested. 

4.  Error  messages  should  alert  students  to  incorrect  performance. 

5.  Data  on  student  study  time  and  responses  are  needed. 
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